Configuring System Settings

Configuring System Settings
 
 
This chapter provides instructions for configuring the following system options:
•
Configuring a Second Management Interface
•
Configuring System Timing
•
Configuring Transmit Timing Source
•
Enabling CLI Timestamping
•
Configuring System Administrative Users
•
Configuring TACACS+ AAA Services for System Administrative Users
•
Configuring Virtual MAC Addresses
•
Configuring Packet Processing and Line Card Availability
•
Configuring Line Card and SPIO Port Redundancy
•
Configuring ASR 5000 Link Aggregation
It is assumed that the procedures to initially configure the system as described in the Getting Started chapter have been completed.
Important: The commands used in the configuration examples in this section are the most likely-used commands and/or keyword options. In many cases, other optional commands and/or keyword options are available. Refer to the Command Line Interface Reference for complete information.
Configuring a Second Management Interface
Refer to Getting Started for instructions on configuring a system management interface on the Switch Processor Input/Output (SPIO) card. This section provides described how to configure a second management interface.
Use the following example to configure a second management interface:
configure
  context local
     interface <interface_name>
        ip address <ipaddress subnetmask>
        exit
     ip route 0.0.0.0 0.0.0.0 <gw_address interface_name>
     exit
  port ethernet <slot#/port#>
     bind interface <interface_name> local
     no shutdown
     media [ rj45 | sfp ]
     end
Notes:
•
For port ethernet slot#, use the actual chassis slot in which the SPIO is installed (slot number 24 or 25).
•
Enter IP addresses using IPv4 dotted-decimal or IPv6 colon-separated notation.
•
For port ethernet port#, use the physical port on the SPIO card that will be used. This is either port 1 or 2. Port 1 represents the top-most port (either RJ-45 or SFP).
•
The SPIO is equipped with dual RJ-45 (copper) and dual SFP (optical fiber) interfaces. The RJ-45 interfaces connect the system to the management network with CAT3 or CAT5 Ethernet cable. The SFP interfaces connect the system to the management network with 1000Base-SX optical fiber cable.
•
Option: In the Ethernet Port configuration mode, configure the port speed, if needed, by entering the medium command. Refer to the Command Line Interface Reference for a complete explanation of this command.
•
In the { ip | ipv6 } route command, other keyword options, instead of the gateway IP address, are available and include: next-hop IP address, point-to-point, and tunnel.
 
Verifying and Saving Your Interface and Port Configuration
Verify that your interface configuration settings are correct by entering the following command:
show ip interface
The output from this command should be similar to that shown below. In this example an interface named mgmt2 was configured in the local context.
Intf Name: mgmt2
Intf Type:       Broadcast
Description:     management2
VRF:             None
IP State: UP (Bound to 24/2)
IP Address: 192.168.100.3 Subnet Mask: 255.255.255.0
Bcast Address: 192.168.100.255 MTU: 1500
Resoln Type: ARP ARP timeout: 60 secs
L3 monitor LC-port switchover: Disabled
Number of Secondary Addresses: 0
Verify that the port configuration settings are correct by entering the following command:
show configuration port <slot#/port#>
slot# is the chassis slot number of the line card where the physical port resides. slot# is either 24 or 25. port# is the number of the port (either 1 or 2). This command produces an output similar to the one shown below; it displays the configuration of port 2 of the SPIO installed in chassis slot 24. In this example, the port is bound to an interface called mgmt2.
config
port ethernet 24/2
description management2
no shutdown
bind interface mgmt2 local
#exit
end
Save your configuration as described in the Saving Your Configuration chapter.
Configuring System Timing
The system is equipped with a clock that supplies the timestamp for statistical counters, accounting records, logging, and event notification. After the initial configuration of the system clock, you can configure the system to communicate with one or more Network Time Protocol (NTP) server(s) to ensure that the clock is always accurate.
In the event of a power outage, the clock is maintained with an accuracy of +/- one minute per month for up to 10 years. This ensures that when power is restored, the system is ready to process sessions and generate accounting, log, and event data with accurate timestamps.
In addition to configuring the timing source, you must configure the system’s time zone.
Setting the System Clock and Time Zone
Use the following command example to configure the system clock and time zone:
clock set <date:time>
configure
  clock timezone <timezone> [ local ]
  end
Notes:
•
Enter the date and time in the format YYYY:MM:DD:HH:mm or YYYY:MM:DD:HH:mm:ss.
•
Refer to the online Help for the clock timezone command for a complete list of supported time zones.
•
The optional local keyword indicates that the time zone specified is the local timezone.
•
Daylight Savings Time is automatically adjusted for time zones supporting it.
Save your configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying and Saving Your Clock and Time Zone Configuration
Enter the following command to verify that you configured the time and time zone correctly:
show clock
The output displays the date, time, and time zone that you configured.
Configuring Network Time Protocol Support
This section provides information and instructions for configuring the system to enable the use of the Network Time Protocol (NTP).
Important: Configure the system clock and time zone prior to implementing NTP support. This greatly reduces the time period that must be corrected by the NTP server.
Use the following example to configure the necessary NTP association parameters:
configure
  ntp
     enable [ <context_name> ]
     server <ip_address>
     end
Notes:
•
context_name is the name of a configured context other than local. Use this option to configure the system to run NTP in a specified context. By default, NTP runs in the local context. This is the recommended configuration.
•
A number of options exist for the server command. Refer to the NTP Configuration Mode Commands chapter in the Command Line Interface Reference for more information.
•
Enter the IP address of NTP servers using IPv4 dotted-decimal or IPv6 colon-separated notation.
Important: Configure the system with at least two NTP servers. It is recommend that four servers are configured.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying the NTP Configuration
Verify the NTP configuration is correct. Enter the following command at the Exec mode prompt:
show ntp associations
The output displays information about all NTP servers. See the output below for an example deploying two NTP servers.
+
-----Peer Selection:
( ) - Rejected/No response
|
 
(X) - False Tick
|
 
(.) - Excess
|
 
(-) - Outlier
|
 
(+) - Candidate
|
 
(#) - Selected
|
 
(*) - System Peer
|
 
(o) - PPS Peer
v
 
 
‌
remote
refid
st
t
when
poll
reach
delay
offset
jitter
===================================================================
+192.68.11.1
192.68.11.55
3
-
677
1024
377
0.800
1.330
1.111
#11.11.1.10
11.11.1.55
3
-
677
1024
377
0.725
-3.134
0.112
‌
The following table describes the parameters output by the show ntp associations command.
Column Title
Description
remote
List of the current NTP servers. One of these characters precedes each IP address to show the server’s current condition:
•
( ) Rejected/No response
•
X False tick
•
. Excess
•
- Outlier
•
+ Candidate
•
# Selected
•
* System peer
•
(o) PPS peer
refid
Last reported NTP reference to which the server is synchronizing.
st
NTP server stratum level.
t
Communication type: broadcast, multicast, etc.
when
Number of seconds since the last contact.
poll
Polling interval between the system and the NTP server.
reach
Octal value of the reachability shift register indicating which responses were received for the previous eight polls to this NTP server.
delay
Round-trip delay (in milliseconds) for messages exchanged between the system and the NTP server.
offset
Number of milliseconds by which the system clock must be adjusted to synchronize it with the NTP server.
jitter
Jitter in milliseconds between the system and the NTP server.
‌
Configuring Transmit Timing Source
This feature is only for application services that use SDH or SONET over Optical or Channelized line cards.
In general, the SPIO automatically provides clocking based on the system clock. However, some application services that use SDH or SONET require greater clocking precision to ensure synchronous transmission. The timing source options include Building Integrated Timing Supply (BITS) and line-timing.
BITS-timing uses Stratum 3-compliant BITS modules resident on the SPIOs.
Line-timing recovers the receive timing from an external clock through a specified port on an Optical or Channelized line card (OLC/OLC2 or CLC/CLC2).
The timing is then distributed via the SPIO to all line cards in the chassis.
Important: To use BITS-timing, the SPIO card must include the optional BITS BNC or 3-pin timing interface. For additional interface information, refer to the Product Overview.
You can enable and configure up to four timing sources: two BITS-timing and two line-timing sources. Having more than one timing source assures redundancy. When enabled BITS-timing always takes priority over line-timing for system clocking.
Configure BITS as the Timing Source
Use the following example to configure BITS as the timing source:
configure
  port bits <slot#/port#>
     mode <e1/t1> framing <type>
     no shutdown
     end
Save the configuration according to the steps in the Verifying and Saving Your Configuration chapter.
Configure Line-timing as the Timing Source
Use the following example to configure line-timing as the timing source:
configure
  port atm <slot#/port#>
     line-timing
     no shutdown
     exit
  port bits <slot#/port#>
     recover line1 <linecard slot #>
     shutdown
     end
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Configure Both BITS and Line as Timing Sources
Use the following example to configure both BITS and line-timing as the timing sources:
configure
  card <CLC slot#>
     framing <mode>
     exit
  port atm <OLC slot#/port#>
     line-timing
     no shutdown
     exit
  port channelized <CLC slot#/port#>
     line-timing
     no shutdown
     exit
  port bits <slot#/port#>
     recover line1 <LC slot#/port#>
     recover line2 <LC slot#/port#>
     no shutdown
     end
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Confirming the Timing Source
Use the show timing command, documented in the Exec Mode Commands chapter of the Command Line Interface Reference, to confirm that the timing source has been configured correctly.
Enabling CLI Timestamping
To display a timestamp (date and time) for every command that is executed on the CLI, enter the following command at the root prompt for the Exec mode:
timestamps
Immediately after you execute the command, the date and time appear.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Configuring System Administrative Users
The Getting Started chapter describes how to configure a context-level security administrator for the system.
This section provides instructions for configuring additional administrative users having the following privileges:
•
Security Administrators: have read-write privileges and can execute all CLI commands, including those available to Administrators, Operators, and Inspectors
•
Administrators: have read-write privileges and can execute any command in the CLI except for a few security-related commands that can only be configured by Security Administrators. Administrators can configure or modify system settings and execute all system commands, including those available to the Operators and Inspectors.
•
Operators: have read-only privileges to a larger subset of the Exec Mode commands. They can execute all commands that are part of the inspector mode, plus some system monitoring, statistic, and fault management functions. Operators do not have the ability to enter the Config Mode.
•
Inspectors: are limited to a few read-only Exec Mode commands. The bulk of these are show commands for viewing a variety of statistics and conditions. An Inspector cannot execute show configuration commands and does not have the privilege to enter the Config Mode.
Configuration instructions are categorized according to the type of administrative user: context-level or local-user.
Important: For information on the differences between these user privileges and types, refer to the Getting Started chapter.
If your deployment does not require the configuration of additional administrative users, proceed to the Configuring Packet Processing and Line Card Availability section.
Configuring Context-level Administrative Users
This section contains information and instructions for configuring context-level administrative user types.
Configuring Context-level Security Administrators
Use the example below to configure additional security administrators:
configure
  context local
     administrator <name> { password <pwd> | encrypted password <pwd> }
     end
Notes:
•
Additional keyword options are available that identify active administrators or place time thresholds on the administrator. Refer to the Command Line Interface Reference for more information about the administrator command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Configuring Context-level Administrators
Use the example below to configure context-level administrators:
configure
  context local
     config-administrator <name> { password <pwd> | encrypted password <pwd> }
     end
Notes:
•
Additional keyword options are available that identify active administrators or place time thresholds on the administrator. Refer to the Command Line Interface Reference for more information about the config-administrator command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Configuring Context-level Operators
Use the example below to configure context-level operators:
configure
  context local
     operator <name> { password <pwd> | encrypted password <pwd> }
     end
Notes:
•
Additional keyword options are available that identify active administrators or place time thresholds on the administrator. Refer to the Command Line Interface Reference for more information about the operator command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Configuring Context-level Inspectors
Use the example below to configure context-level inspectors:
configure
  context local
     inspector <name> { password <pwd> | encrypted password <pwd> }
     end
Notes:
•
Additional keyword options are available that identify active administrators or place time thresholds on the administrator. Refer to the Command Line Interface Reference for more information about the inspector command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying Context-level Administrative User Configuration
Verify that the configuration was successful by entering the following command:
show configuration context local
This command displays all of the configuration parameters you modified within the Local context during this session. The following displays sample output for this command. In this example, a security administrator named testadmin was configured.
config
context local
interface mgmt1
ip address 192.168.1.10 255.255.255.0
#exit
subscriber default
#exit
administrator testadmin encrypted password fd01268373c5da85
   inspector testinspector encrypted password 148661a0bb12cd59
#exit
port ethernet 24/1
bind interface mgmt1 local
#exit
end
Configuring Local-User Administrative Users
Use the example below to configure local-user administrative users:
configure
  local-user username <name>
  end
Notes:
•
Additional keyword options are available identify active administrators or place time thresholds on the administrator. Refer to the Command Line Interface Reference for more information about the local-user username command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying Local-User Configuration
Verify that the configuration was successful by entering the following command:
show local-user verbose
This command displays information on configured local-user administrative users. A sample output for this command appears below. In this example, a local-user named SAUser was configured.
Username: SAUser
Auth Level: secadmin
Last Login: Never
Login Failures: 0
Password Expired: Yes
Locked: No
Suspended: No
Lockout on Pw Aging: Yes
Lockout on Login Fail: Yes
Configuring TACACS+ AAA Services for System Administrative Users
This section describes TACACS+ (Terminal Access Controller Access Control System+) AAA (Authentication Authorization and Accounting) service functionality and configuration on the ASR 5000.
Operation
TACACS+ is a secure, encrypted protocol. By remotely accessing TACACS+ servers that are provisioned with the administrative user account database, the ASR 5000 can provide TACACS+ AAA services for system administrative users. TACACS+ is an enhanced version of the TACACS protocol, and uses TCP instead of UDP.
The ASR 5000 system serves as the TACACS+ Network Access Server (NAS). As the NAS the system requests TACACS+ AAA services on behalf of authorized system administrative users. For the authentication to succeed, the TACACS+ server must be in the same local context and network accessed by the system.
The system supports TACACS+ multiple-connection mode. In multiple-connection mode, a separate and private TCP connection to the TACACS+ server is opened and maintained for each session. When the TACACS+ session ends, the connection to the server is terminated.
TACACS+ is a system-wide function on the ASR 5000. TACACS+ AAA service configuration is performed in TACACS Configuration Mode. Enabling the TACACS+ function is performed in the Global Configuration Mode. The system supports the configuration of up to three TACACS+ servers.
Once configured and enabled on the system, TACACS+ authentication is attempted first. By default, if TACACS+ authentication fails, the system then attempts to authenticate the user using non-TACACS+ AAA services, such as RADIUS.
User Account Requirements
Before configuring TACACS+ AAA services on the ASR 5000, note the following TACACS+ server and system user account provisioning requirements:
TACACS+ User Account Requirements
The TACACS+ server must be provisioned with the following TACACS+ user account information:
•
A list of known administrative users.
•
The plain-text or encrypted password for each user.
•
The name of the group to which each user belongs.
•
A list of user groups.
•
TACACS+ privilege levels and commands that are allowed/denied for each group.
Important: TACACS+ privilege levels are stored as Attribute Value Pairs (AVPs) in the network’s TACACS+ server database. Users are restricted to the set of commands associated with their privilege level. A mapping of TACACS+ privilege levels to ASR 5000 CLI administrative roles and responsibilities is provided in the table below.
Mapping of TACACS+ Privilege Levels to CLI Administrative Roles
TACACS+ Privilege Level
CLI Administrative Access Privileges
CLI
FTP
ECSEMS
Lawful Intercept
CLI Role
0
Yes
No
No
No
Inspector
1
Yes
No
Yes
No
Inspector
2
No
Yes
No
No
Inspector
3
Yes
Yes
No
No
Inspector
4
Yes
Yes
Yes
No
Inspector
5
Yes
No
No
No
Operator
6
Yes
No
Yes
No
Operator
7
No
Yes
No
No
Operator
8
Yes
Yes
No
No
Operator
9
Yes
Yes
Yes
No
Operator
10
Yes
No
No
No
Administrator
11
Yes
No
Yes
No
Administrator
12
No
Yes
No
No
Administrator
13
Yes
Yes
No
Yes
Administrator
14
Yes
Yes
Yes
No
Administrator
15
Yes
Yes
Yes
Yes
Administrator
‌
User Account Requirements
TACACS+ users who are allowed administrative access to the system must have the following user account information defined on the ASR 5000:
•
username
•
password
•
administrative role and privileges
Important: For instructions on defining users and administrative privileges on the system, refer to the Configuring System Administrative Users chapter in this guide.
Configuring TACACS+ AAA Services
This section provides an example of how to configure TACACS+ AAA services for administrative users on the system.
Caution: When configuring TACACS+ AAA services for the first time, the administrative user must use non-TACACS+ services to log in to the ASR 5000. Failure to do so will result in the TACACS+ user being denied access to the system.
Log in to the system using non-TACACS+ services.
Use the example below to configure TACACS+ AAA services on the system:
configure
   tacacs mode
      server priority <priority_number> ip-address <tacacs+srvr_ip_address>
      end
Note:
•
server priority <priority_number>: Must be a number from 1 to 3, that specifies the order in which this TACACS+ server will be tried for TACACS+ authentication. 1 is the highest priority, and 3 is the lowest.
•
ip-address: Must be the IPv4 address of a valid TACACS+ server that will be used for authenticating administrative users accessing this system via TACACS+ AAA services.
•
By default, the TACACS+ configuration will provide authentication, authorization, and accounting services.
Enable TACACS+ on the ASR 5000:
configure
   aaa tacacs+
   end
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Important: For complete information on all TACACS+ Configuration Mode commands and options, refer to the TACACS Configuration Mode Commands chapter in the Command Line Interface Reference.
Verifying the TACACS+ Configuration
This section describes how to verify the TACACS+ configuration:
Log out of the system CLI, then log back in using TACACS+ services.
Important: Once TACACS+ AAA services are configured and enabled on the ASR 5000, the system first will try to authenticate the administrative user via TACACS+ AAA services. By default, if TACACS+ authentication fails, the system then continues with authentication using non-TACACS+ AAA services.
At the Exec Mode prompt, enter the following command:
show tacacs
The command output provides summary information for each active TACACS+ session such as username, login time, login status, current session state and privilege level.
A example of this command’s output is provided below. In this example, a system administrative user named asradmin has successfully logged in to the system via TACACS+ AAA services.
active session #1:
   login
username : asradmin
   login
tty : /dev/pts/1
   time
of login : Fri Oct 22 13:19:11 2011
   login
server priority : 1
   current
login status : pass
   current
session state : user login complete
   current
privilege level : 15
   remote
client application : ssh
   remote
client ip address : 111.11.11.11
   last
server reply status : -1
total TACACS+ sessions
: 1
Important: For details on all TACACS+ maintenance commands, refer to the show tacacs chapter in the Statistics and Counters Reference.
Configuring Virtual MAC Addresses
When you enable virtual MAC addressing, a single block of 256 addresses is added to the system configuration. The MAC addresses assigned and stored in the EPROM on Ethernet Line Cards are disregarded; MAC addresses for all ports on all Ethernet Line Cards are assigned from the specified block of virtual MAC addresses. This does not affect the MAC addresses on SPIO cards.
As in normal MAC address assignments, the corresponding ports on the upper and lower line cards have the same assigned MAC address. When you enable virtual MAC addressing, these addresses are all assigned from the specified block of 256 addresses.
If you enable virtual MAC addressing and remove a line card from the system, MAC addresses do not have to be reassigned because the MAC addresses in use do not belong to any line card. Therefore, if a line card is removed from the system, there is no possibility that any port on a line card in the system is using any of the MAC addresses that belong to the removed line card.
Use the following example to configure virtual MAC addressing:
configure
  port mac-address virtual-base-address <MAC_Address>
  end
Notes:
•
MAC_Address is the first address of a block of 256 MAC addresses. The system has reserved 65536 MAC addresses (00:05:47:FF:00:00 to 00:05:47:FF:FF:FF) for use by customers. This range allows you to create 256 address blocks each containing 256 MAC addresses (for example, 00:05:47:FF:00:00, 00:05:47:FF:01:00, 00:05:47:FF:02:00, 00:05:47:FF:03:00, 00:05:47:FF:04:00, etc.).
Caution: This configuration requires a valid block of unique MAC addresses that are not used anywhere else. The use of non-unique MAC addresses can degrade and impair the operation of your network.
Save your configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying Virtual MAC Address Configuration
Verify port information by entering the following command
show port info <slot#/port#>
slot# is the chassis slot number of the line card on which the physical port resides. port# is the physical port on the line card.
The output of this command should be similar to that shown in the example below.
Port: 36/8
Port Type : 10/100 Ethernet
Description : (None Set)
Controlled By Card : 4 (Packet Accelerator Card)
Redundancy Mode : Port Mode
Redundant With : 20/8
Preferred Port : Non-Revertive
Physical ifIndex : 604504064
Administrative State : Enabled
Configured Duplex : Auto
Configured Speed : Auto
MAC Address : 00-05-47-02-04-3F
Link State : Up
Link Duplex : Full
Link Speed : 100 Mb
Logical ifIndex : 604504065
Operational State : Down, Standby
Configuring Packet Processing and Line Card Availability
As discussed in the Understanding the System Boot Process section of the Understanding System Operation and Configuration chapter, when the system boots up, all installed packet processing cards are placed into standby mode. You must activate some of these cards in order to configure and use them for session processing. Others may remain in standby mode to serve as redundant components.
When you activate an application card, the line card behind it shows up as attached and in a Ready state. Only when you bind a logical interface to one of the ports of the line card pair will the line cards assume an active and standby state.
This section provides instructions for activating packet processing cards and specifying their redundancy.
Important: Refer to the Product Overview Guide for information about system hardware configurations and redundancy.
Enter the following command to check the application card’s operational status:
show card table
This command lists the packet processing cards and RCCs installed in the system by their slot number, their operational status, whether or not the card is a single point of failure (SPOF), and its attachment to a line card.
Check the line card operational status by entering the following command:
show linecard table
This command lists the line cards installed in the system by their slot number, their operational status, whether or not the card is a single point of failure (SPOF), and its attachment to a packet processing card or SMC.
Use the following example to configure packet processing card and line card availability:
configure
  card <slot_#>
     mode active { pac | psc }
     exit
  card-standby-priority <slot#_p1 slot#_p2 ... slot#_pn>
  end
Notes:
•
When activating cards, remember to keep at least one card in standby mode for redundancy.
•
Repeat for every other packet processing card in the chassis that you wish to activate.
•
The card-standby-priority specifies the order in which the system will use standby packet processing cards as redundant components.
•
By default, the system uses the standby packet processing card in the highest-numbered slot (slot 16) as the first card to use for redundancy. This step is required if there are processing cards installed in the system that are in standby mode, and you want to configure the system to use an order other than the default.
•
slot#_p1 is the chassis slot number of the standby packet rpocessing card that you want to use first as a redundant component. slot#_p2 is the chassis slot number of the standby processing card that you want to use second as a redundant component. slot#_pn is the chassis slot number of the standby packet processing card that you want to use as the last redundant component.
Example
A system has three packet processing cards that are in standby mode. They are installed in chassis slots 14, 15, and 16. If an active processing card fails, and you want the packet processing card in slot 15 to replace the failed packet rpocessing card followed by the packet processing card in slot 14, enter the following command:
card-standby-priority 15 14
In the unlikely event that the packet rpocessing cards in chassis slots 15 and 14 are unavailable, the system automatically uses the remaining standby packet processing card in slot 16 for redundancy.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying Packet Processing and Line Card Configurations
Verify that the configuration was successful. Depending on the type of card(s) you activated, enter either or both of the following commands:
show card table
show linecard table
Any card that you made active should now have an operational status of Active.
 
Configuring Line Card and SPIO Port Redundancy
Port redundancy for line cards and SPIOs provides an added level of redundancy that minimizes the impact of network failures that occur external to the system. Examples include switch or router port failures, disconnected or cut cables, or other external faults that cause a link down error.
Caution: To ensure that system line card and port-level redundancy mechanisms function properly, disable the Spanning Tree protocol on devices connected directly to any system port. Failure to turn off the Spanning Tree protocol may result in failures in the redundancy mechanisms or service outage.
By default, the system provides port-level redundancy when a failure occurs, or you issue the port switch to command. In this mode, the ports on active and standby line cards (for example, 17/1 and 33/1) have the same MAC address, but since only one of these ports may be active at any one time there are no conflicts. This eliminates the need to transfer MAC addresses and send gratuitous ARPs in port failover situations. Instead, for Ethernet ports, three Ethernet broadcast packets containing the source MAC address are sent so that the external network equipment (switch, bridge, or other device) can re-learn the information after the topology change. However, if an line card removal is detected, the system sends out gratuitous ARPs to the network because of the MAC address change that occurred on the specific port.
With port redundancy, if a failover occurs, only the specific port(s) become active. For example; if port 17/1 fails, then port 33/1 becomes active, while all other active ports on the line card in slot 17 remain in the same active state. In port failover situations, use the show port table or show linecard table commands to check that ports are active on both cards and that both cards are active.
Take care when administratively disabling a port that is one of a redundant pair. A redundant pair comprises both the active and standby ports—for example 17/1 and 33/1. If 17/1 is active, administratively disabling 17/1 through the CLI does not make 33/1 active. It disables both 17/1 and 33/1 because an action on one port has the same effect on both. Refer to Enabling Line Card and SPIO Redundancy below and Creating and Configuring Ethernet Interfaces and Ports in the System Element Configuration Procedures chapter.
If card-level redundancy is initiated, there is no port-level redundancy in a line card or SPIO failover. The standby line card or SPIO becomes active and all ports on that card become active. With line cards, the system automatically copies all the MAC addresses and configuration parameters used by the failed line card to its redundant counterpart. The ports on SPIOs keep their original MAC addresses, and the system automatically copies the failed SPIO’s configuration parameters to its redundant counterpart. The packet processing card automatically re-routes to its redundant line card.
With the SPIO cards, any time there is a port or card switch gratuitous ARPs are sent.
Important: Be aware that in the case of a system with only one SMC and two SPIO cards, both SPIOs come up online. Automatic switching of Ethernet ports does not occur in this scenario, but you can initiate card and port switching by using the card spio switch to and port switch to commands.
Port redundancy can be configured to be revertive or non-revertive. With revertive redundancy service is returned to the original port when service is restored.
This feature requires specific network topologies to work properly. The network must have redundant switching components or other devices that the system is connected to. The following diagrams show examples of a redundant switching topologies and how the system reacts to various external network device scenarios.
 
Network Topology Example Using Line Card Port Redundancy
Port Redundancy Failover in Cable Defect Scenario
In the example above, an Ethernet cable is cut or unplugged, causing the link to go down. When this event occurs, the system, with port-mode redundancy enabled, recognizes the link down state and makes port 33/1 the active port. The switching devices, using some port redundancy scheme, recognizes the failure and enables the port on the secondary switch that the line card in slot 33 is connected to, allowing it to redirect and transport data.
Port Redundancy Failover in External Network Device Failure Scenario
In the example above, a switch failure causes a link down state on all ports connected to that switch. This failure causes all redundant ports on the line card in slot 33 to move into the active state and utilize the redundant switch.
Enabling Line Card and SPIO Port Redundancy
Use the following example to enable port redundancy:
configure
  card <slot_#>
     redundancy { card-mode | mixed-mode | port-mode }
     end
Notes:
•
The card-mode keyword indicates that no port redundancy is used. The system provides card-level redundancy, which is triggered by an internal failure. The port-mode keyword, available for Ethernet and SPIO line cards, indicates that port redundancy will be enabled. This is the default redundancy mode.
Important: You do not need to use this configuration for each line card or SPIO. The system intuitively understands that if the command is entered for an active line card, the standby line will operate in the same mode. For example, if you enter the command for the line card in slot 17, it automatically places the line card in Slot 33 into port redundant operation.
Important: If you network-boot a dual-SMC chassis with SPIO port redundancy enabled, you should have CFE1.1.0 or greater in flash on both SMCs. Otherwise, you risk having a standby SMC that can't boot from the network in certain circumstances. You can use any version of the CFE with SPIO port redundancy if the SMCs boot from a local file system (/flash, /pcmcia1, or /pcmcia2).
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
 
Verifying Line Card and SPIO Port Redundancy
View the configuration of the card by entering the following command:
show configuration card <slot_#>
slot_# is the chassis slot number where the line card or SPIO you want to configure is installed.
The following is a sample of output for an line card in slot 17 and a SPIO in slot 24 that both have redundancy enabled.
[local]host_name# show config card 17
config
card 17
redundancy port-mode
#exit
end
[local]host_name# show config card 24
config
card 24
redundancy port-mode
#exit
end
Configuring Line Card and SPIO Port Redundancy Auto-Recovery
When port redundancy is enabled at the card level, you can configure a port auto-recovery feature. When a port failure occurs and the preferred port is returned to service (link is up), control is automatically returned to that port. By default, ports are in a non-revertive state, meaning that no ports are preferred, requiring a manual port switch to return use to the original port.
Important: This feature is applied on a per port basis, allowing you to configure specific ports to be used on individual line cards or SPIOs. For example, you could configure ports 1 through 4 as preferred on the line card in slot 17, and configure ports 5 through 8 as the preferred ports on the line card in slot 33. On a SPIO, you could configure port 1 as preferred on the SPIO in slot 24 and configure port 2 as preferred on the SPIO in slot 25. In this scenario, both line cards or SPIOs would be in an active state while providing line card and port redundancy for the other.
Use the following example to configure a preferred port for revertive, automatic return to service when a problem has cleared:
configure
  port ethernet <slot#/port#>
     preferred slot <slot#>
     end
Notes
•
If you do not specify a preference, redundancy is non-revertive. If you do specify a preference, redundancy is revertive to the specified card.
•
Repeat for each additional port that you want to make preferred.
Caution: A preference cannot be configured in normal redundancy mode. Attempting to do so will produce an error message from the cli command.
Save the configuration as described in the Verifying and Saving Your Configuration chapter.
Verifying Line Card and SPIO Port Redundancy Auto-Recovery
Verify port information by entering the following command
show port info <slot#/port#>
slot# is the chassis slot number of the line card on which the physical port resides.
port# is the physical port on the line card.
The following shows a sample output of this command for port 1 on the LC in slot 17:
[local]host_name# show port info 17/1
Port: 17/1
Port Type : 10/100 Ethernet
Description : (None Set)
Controlled By Card : 1 (Packet Accelerator Card)
Redundancy Mode : Port Mode
Redundant With : 33/1
Preferred Port : Revertive to port 17/1
Physical Index : 285278208
Administrative State : Disabled
Configured Duplex : Auto
Configured Speed : Auto
MAC Address : 00-05-47-01-11-00
Link State : Up
Link Duplex : Unknown
Link Speed : Unknown
Logical ifIndex : 285278209
Operational State : Down, Active
Configuring ASR 5000 Link Aggregation
A Link Aggregation Group (LAG) works by exchanging control packets via Link Aggregation Control Protocol (LACP) Marker Protocol over configured physical ports with peers to reach agreement on an aggregation of links as defined in IEEE 802.3ad. The LAG sends and receives the control packets directly on physical ports attached to different XGLC (10 Gig Ethernet) or QGLC (Quad Ethernet 1000) cards.
Link aggregation (also called trunking or bonding) provides higher total bandwidth, auto-negotiation, and recovery by combining parallel network links between devices as a single link. A large file is guaranteed to be sent over one of the links, which removes the need to address out-of-order packets.
A LAG can be formed by configuring:
•
Ports from the same QGLC
•
Ports from top-bottom QGLCs with port redundancy
•
Ports from side-by-side XGLCs
•
Ports from horizontal QGLCs where ports could be from different cards, while some cards could be in L2 (side-by-side) redundancy.
Important: LAG cannot be configured when XGLCs have been configured for side-by-side redundancy. A failure message appears on the CLI when such an attempt is made.
LAG and Master Port
Logical port configurations (VLAN and binding) are defined in the master port of the LAG. If the master port is removed because of a card removal/failure, another member port becomes the master port (resulting in VPN binding change and outage), unless there is a redundant master port available.
LAG and Port Redundancy
A LAG supports redundant ports, either top-down on the QGLC (vertical link aggregation) or horizontal link aggregation on the XGLC, when only one port is active in the pair. By default, active ports in a LAG can be on any XGLC or QGLC, but optionally, all ports in a LAG can be auto-switched to another card when certain active port counts or bandwidth thresholds are crossed.
LAG and Multiple Switches
This feature connects ports on XGLCs or QGLCs to ports on Ethernet switches. A port failure/switch forces all ports in a LAG to switch to another XGLC or QGLC when a specified threshold is crossed. This works in a way similar to the auto-switch feature for port redundancy. LACP runs between the ASR 5000 and the Ethernet switch, exchanging relevant pieces on information, such as health status.
The following table summarizes the functionality of LAG with QGLC and XGLC cards.
QGLC and XGLC LAG Functionality
Card Type
ASR 5000
LAGID
Ethernet Switch A
Ethernet Switch B
QGLC
Port 1
1
Port 1
----
Port 2
1
Port 2
----
Port 3
1
----
Port 1
XGLC
Slot 1
1
Port 1
----
Slot 2
1
----
Port 1
‌
Multiple Switches without L2 Redundancy
If one LAG is connected to different peers, by default, the implementation selects the higher bandwidth peer to form aggregation. If there are several horizontal cards with LAG ports that are all in active mode (no L2 redundancy) connected to different switches, each card provides a candidate aggregation of bandwidth. Selection by bandwidth works because the failure of one port causes that card’s bandwidth to be lower, thus causing another card to be selected.
The figure below shows an LAG established across two line card ports without L2 redundancy.
LAG without L2 Redundancy, Two Ethernet Switches
 
Multiple Switch with L2 Redundancy or Active-Active Mode
To handle the implementation of Link Aggregation Control Protocol without requiring standby ports to pass LACP packets, two separate instances of LACP are started on redundant cards. The two LACP instances and port link state are monitored to determine whether to initiate an auto-switch (including automatic L2 port switch).
Two switches can also be connected to odd and even slots of an XGLC in active-active mode without L2 redundancy. Two LACP instances are started for odd and even slots, and similar monitoring and switching occurs.
The figure below shows an LAG established across two line card ports with L2 redundancy.
LAG with L2 Redundancy, Two Ethernet Switches
An LACP implementation with L2 redundancy cannot pass traffic even though standby ports have link up. For example, with two QGLC cards connected to two different Ethernet switches configured as a top-bottom pair and all ports in the same LAG, failure of ports would not trigger a LAG switch until the active port number ratio flipped (more ports down than up).
To handle this case without requiring standby ports to pass LACP packets, separate instances of LACP are started on redundant cards. The LAG manager monitors the two LACP instances and port link states to decide whether to auto-switch (including automatic L2 port switching).
Port States for Auto-Switch
Ports are classified in one of the four states (shown in the following table) to determine whether to start auto-switching.
For counters, State(x) represents the number of ports on a card in that state.
Auto-Switch Port States
State
Counter
Description
Link
L(x)
Physical link up
Standby
S(x)
Link up but in standby mode
Waiting
W(x)
Waiting for Link Aggregation Control Protocol negotiation
Aggregated
A(x)
Aggregation formed
‌
Hold Time
Once the LAG manager switches to another LACP instance, it does not consider another change for a short period to let link and LACP negotiation settle down. This “hold time” is configurable.
The LAG manager also enters/extends the hold period when an administrator manually switches ports to trigger a card switch.
Preference and Revertive Mode
You can define which card is preferred to implement revertive mode. This preference is defined per LAG group. Port preference is not allowed in this mode.
 
Auto-Switch Criteria
The following criteria determine the switching of card x to card y to provide better bandwidth while allowing manual intervention. The evaluation of the criteria occurs outside of the hold period.
Ports are automatically switched from card x to card y when A(y) ? = 1, at least one port is in aggregated state on card y, and one of the following conditions is true (in order of precedence):
•
L(x) > L(y) Less ports with link Up on card x than card y
•
S(x) > S(y) More ports in Standby state on card x than card y
•
W(x) > W(y) More ports in Waiting state on card x than card y
•
A(x) < A(y) Fewer ports in Aggregated state on card x than card y
•
Card y is preferred
•
Card y is selected.
Distribution Options
The specified distribution option controls how the LAG hash map is generated. This option is set on the master port for use by the whole LAG via the link-aggregation distrbution command. The following table identifies the distribution options (assuming port index 0,1,2,3 were selected):
LAG Distribution Options
Distribution
Meaning
Example
Simple
Repetition of all selected port indices
0123012301230123…
Rotate
Repetition of the rotated port indices
0123123023013012…
Block
Blocks of the same port index
0000111122223333
Random
Based on pseudo random number
—
‌
QGLC Link Aggregation
The aggregated ports must be on the same QGLC redundant pair. Link aggregation does not work across line card slots. In the event of a failure of one or more of the member physical ports, the remaining ports continue to be aggregated. Top and bottom QGLC cards can be connected to different switches in a LAG.
Requirements
Observe the following requirements:
•
Assure that links between the two systems are full duplex and at the same speed.
•
Set the port medium configuration to auto or full duplex and maximum speed.
•
An aggregation group can consist of from one to four ports. A port can only be in one aggregation group; for example, Port 3 can be in Group A linked to Switch 1, but it cannot simultaneously be in Group B linked to Switch 2.
•
Certain physical port configuration changes, such as the MAC address or SRP, are prohibited on any interface participating in link aggregation
There is more on configuring ports and port redundancy in Configuring Line Card and SPIO Port Redundancy.
Operation
Link aggregation operates as a sublayer between the MAC client and the MAC layer.
Each MAC passes received frames up for control or collection in an aggregator—a logical MAC that aggregates several links together. The MAC client sends frames to the aggregator for distribution among MACs, as follows:
QGLC Link Aggregation Traffic Flow
The aggregator and each MAC share the same MAC address, which means the MAC has no need to parse two different unicast MAC addresses.
Frame distribution uses an algorithm to distribute frames among MACs that prevents both the mis-ordering of frames belonging to the same “conversation,” and frame duplication.
XGLC Link Aggregation
Because the XGLC is a full-height card that requires top and bottom card slots, link aggregation takes place horizontally within ports on different XGLCs.
Link Aggregation Control
One port in an aggregation group is configured as a master so that all traffic (except control traffic) in the aggregation group logically passes through this port. It is recommended (although not required) that you set up the master first by CSP (task managing card/slot/ports), and unset last.
The following command creates link aggregation group N with port slot#/port# as master. Only one master port is allowed for a group. N must be in the range of [1...1023].
configure
  port ethernet <slot#/port#>
     link-aggregation master group <N>
     exit
Important: Link Aggregation Control Protocol (LACP) starts running only when the Master port is enabled.
Use the following command to add a port as member of link aggregation group number N only if the master port is assigned. Otherwise, it is added to the group when the master port is assigned:
  port ethernet <slot#/port#>
     link-aggregation member group <N>
     exit
Important: The VPN can only bind the master port, and a VLAN can only be created on the master port. VPN CLI and vpnmgr return a failure message if you attempt to bind to a link aggregation member port.
Two redundant line cards and their controlling packet services card function as a system; this allows loopback addressing between vertical slots. Each system that participates in link aggregation has a unique system ID that consists of two bytes priority (where the lowest number (0) has the highest priority) and six bytes of MAC derived from the first port’s MAC address. The following command sets the system priority used to form the system ID. P is a hex in the range [0x0000..0xFFFF]. The default is 0x8000.
  card <slot#>
     link-aggregation system-priority <P>
Ports in a system are assigned keys. The group number maps directly to the key, whereupon only ports with the same key can be aggregated. Ports on each side of the link use a different aggregation key.
The system ID, port key and port ID of two peers form the Link Aggregation Group Identifier (LAGID). You can aggregate links having the same LAGID. Systems are often configured initially with each port in its own aggregation (requiring a separate key per port), or with all ports in the same aggregation (a single key for all ports). Negotiation via LACP would qualify the actual aggregation.
Systems exchange information about system ID, port key and port ID with peers across the physical links using LACP.
LACP packets are defined with the Slow Protocol format. Each system sends out its own (“actor”) information and its last received information about its peer (“partner”) over the physical link.
Use the following commands to set the LACP parameters. LACP can run in active mode to send LACP packets periodically, or in passive mode, in which it only responds to LACP packets it receives.
LACP can send packets at either a slow (30s) or fast (1s) rate. The defaults for this release are Active and Slow; see the sample configuration below:
config
  port ethernet <slot#/port#>
     link-aggregation lacp active rate fast
Peers send out LACP packets when the state changes or if a difference is found from a received LACP packet about its own state.
Corresponding ports on a QGLC or XGLC redundant pair cannot be active at the same time. Redundant ports share the same MAC address, so after a failover is resolved, the original port rejoins the link aggregation group.
Redundancy Options
For redundancy, there is an option that controls the auto-switching of L2 redundant or active-active ports when they are connected to two switches. Set this option on the master port for use with the whole group
link aggregation [redundancy { link aggregation toggle link | switched}] [hold-time <sec>] [preferred slot {card_number | none}
Distribution Options
This option controls how a LAG hash map is generated. This method is required in the case of ECMP over LAG. Set this option on the master port for use with the whole group.
link aggregation {simple | rotate | block | random}
The following list defines the distribution options (assuming port index 0,1,2,3 were selected.
•
simple: Repetition of all selected port indexes (Example: 0123012301230123…)
•
rotate: Repetition of rotated port index (Example: 0123123023013012…)
•
block: Blocks of the same port index (Example: 0000111122223333)
•
random: Based on pseudo random number
Horizontal Link Aggregation with Two Ethernet Switches
When a LAG contains two sets of ports each connecting to a different switch, the operator has the ability to specify the slot/port (connected to the destination switch) when switching ports.
The exec mode link-aggregation port switch to <slot/port> command is used to configure this option. The slot/port is any valid port connected to the destination switch. The following criteria apply to the setting of this option:
•
slot/port must support LAG.
•
slot/port must be configured with LAG.
•
slot/port must not be already actively distributing
•
slot/port must have negotiated a link aggregation partner in standard mode.
•
slot/port's partner must have an equal or higher in standard mode.
•
slot/port's partner bundle must have equal or higher bandwidth in standard mode.
•
Switching to slot/port must not violate preference within hold-time in standard mode.
Link Aggregation Status
To check the status of link aggregation, use the following commands:
•
show port table
•
show port info <slot>/<port>
 
 
Cisco Systems Inc.
Tel: 408-526-4000
Fax: 408-527-0883